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Uncertainty  and  Sensitivity  of 
Ecosystem  Restoration  Decisions:  A 
Case  Study  from  Coastal  Louisiana 

by  S.  Kyle  McKay1  and  J.  Craig  Fischenich2 


OVERVIEW:  Effective  ecosystem  restoration  decision  making  requires  analysis  of  the  costs  and 
benefits  of  alternative  actions.  The  outcomes  of  restoration  actions  are  often  uncertain  -  and  the 
models  used  to  assess  the  outcomes  are  equally  uncertain  -  due  to  incomplete  ecological 
knowledge  and  other  factors,  such  as  limited  data.  These  uncertainties  can  impose  some  risk. 
This  technical  note  provides  an  overview  of  two  quantitative  methods  for  assessing  risk 
associated  with  restoration  decisions,  sensitivity,  and  uncertainty  analysis.  An  example  of  these 
methods  is  provided  for  a  Louisiana  coastal  wetland  restoration  study. 

RISK  ASSESSMENT:  A  risk  is  a  potential  adverse  consequence  that  may  (or  may  not)  be 
realized  in  the  future  and  is  typically  defined  by  the  product  of  the  likelihood  of  an  outcome  (i.e., 
probability)  and  the  consequence  of  that  outcome  (Suedel  et  al.  2012).  Forecasting  the  outcome 
of  a  restoration  project  often  includes  considerable  uncertainty  due  to  incomplete  knowledge, 
imperfect  models,  stochastic  environmental  conditions,  and  many  other  factors.  Risk-informed 
decision  making  requires  explicit  acknowledgement  of  key  sources  of  uncertainty  and  seeks  to 
minimize  adverse  outcomes  related  to  those  uncertainties. 

Risk  assessment  is  a  field  of  study  that  applies  an  array  of  qualitative  and  quantitative  tools  to 
define  likelihoods  and  outcomes  of  a  given  decision  (Suedel  et  al.  2012).  Assessment  of  project 
risk  and  uncertainty  has  been  required  of  US  Army  Corps  of  Engineers  (USACE)  planners  since 
the  establishment  of  Principles  and  Standards  in  1973.  Qualitative  risk  assessment  methods 
include  listing  sources  of  risk  and  uncertainty,  relative  rankings  of  risks,  and  multi-objective 
comparison  of  risks.  Although  qualitative  methods  have  been  used  in  restoration  decision 
making,  few  quantitative  risk  assessments  have  been  undertaken  (Suedel  et  al.  2012).  This 
disparity  is  partly  due  to  the  complexities  in  scoping  and  conducting  quantitative  risk 
assessments.  Here,  the  authors  focus  on  two  quantitative  risk  assessment  methods  (sensitivity 
and  parametric  uncertainty  analysis)  and  demonstrate  their  application  in  the  planning  of  a  large- 
scale  coastal  wetland  restoration  project. 

Sensitivity  Analysis.  The  risk  of  a  given  decision  may  be  defined  by  plausible  future  scenarios, 
the  outcome  for  each  scenario,  and  the  probability  of  each  scenario  occurring  (Schultz  et  al.  2010). 
When  elements  of  the  future  are  uncertain  or  incomplete  (e.g.,  climate  change,  land  use,  invasive 
species),  different  scenarios  can  be  analyzed  in  an  effort  to  capture  this  uncertainty  and  inform 
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decision  making.  Analyzing  model  outcomes  for  multiple,  plausible,  future  scenarios  provides  one 
technique  for  assessing  risks  and  increases  the  overall  chances  of  project  success  (Suedel  et  al. 

2012) .  For  instance,  a  team  may  identify  three  plausible  future  scenarios  such  as  low,  moderate, 
and  high  development  at  an  urban  project  site.  If  alternatives  at  this  site  are  consistently  ranked 
across  all  scenarios,  then  the  team  may  be  more  confident  in  the  robustness  of  their  decision. 

Sensitivity  analysis  is  the  systematic  testing  of  model  outcomes  to  changes  in  parameter  values 
(Schmolke  et  al.  2010).  These  analyses  are  often  undertaken  to  better  understand  the  impact  of 
model  assumptions  or  parameterization  (e.g.,  Convertino  et  al.  2012,  McKay  and  Fischenich 

2013) .  Scenario  analysis  is  a  specific  form  of  sensitivity  analysis.  In  this  case,  a  project  team  is 
assessing  the  sensitivity  of  the  decision  (rather  than  the  sensitivity  of  the  model)  to  changes  in 
model  parameterization. 

Uncertainty  Analysis.  At  the  basal  level,  uncertainty  is  a  lack  of  knowledge  (Schultz  et  al. 
2010).  Numerous  alternative  typologies  exist  to  characterize  sources  of  uncertainty.  Ascough  et 
al.  (2008)  present  the  following  types  of  uncertainty: 

•  Knowledge  or  epistemic  uncertainty:  current  limitation  of  knowledge,  which  may  be  reduced 
by  additional  data  and  research 

•  Variability  uncertainty:  inherent  variability  or  stochasticity  with  a  system 

•  Decision-making  uncertainty:  ambiguity  regarding  the  quantification  of  objectives 

•  Linguistic  uncertainty:  vague,  ambiguous,  and/or  context-dependent  language 

The  authors  focus  herein  on  the  most  reducible  form  of  uncertainty,  epistemic.  Knowledge  of 
ongoing  processes  may  be  limited  by  incomplete  scientific  understanding.  Process  understanding 
is  often  poorly  understood  and  understated  until  additional  data  are  brought  to  bear  (i.e.,  “there 
are  things  we  do  not  know  we  don’t  know”1).  Epistemic  uncertainty  also  arises  in  modeling  from 
the  following  sources. 

•  Parametric  uncertainty:  data  uncertainty  arising  from  measurement  error,  small  sample  size, 
analytical  techniques  applied,  and  presentation  methods  (e.g.,  incomplete  spatial  and 
temporal  data  collection  and  subsequent  reduction  to  a  mean  and  standard  deviation) 

•  Structural  uncertainty:  inadequate  representation  of  the  system  or  multiple  alternative 
representations  (e.g.,  linear  v.  exponential  regressions  with  similar  support) 

•  Technical  uncertainty:  hardware  and  software  errors  (e.g.,  bugs,  rounding) 

EXAMPLE  APPLICATION  -  WETLAND  RESTORATION  IN  COASTAL  LOUISIANA: 

Since  1978,  Coastal  Louisiana  has  lost  marsh  at  an  average  rate  of  over  30  km  /yr  due  to 
river/floodplain  disconnection,  sea  level  rise,  subsidence,  coastal  erosion,  and  other  factors 
(Barras  et  al.  2008).  In  the  fall  of  2005,  Hurricanes  Katrina  and  Rita  awakened  the  United  States’ 
public  to  the  natural  protection  and  ecological  value  coastal  wetlands  provide  in  reducing  effects 
of  hurricanes  on  coastal  communities.  In  response  to  these  catastrophic  events,  the  US  Congress 
directed  the  USACE  to  “conduct  a  comprehensive  hurricane  protection  analysis  and  design... to 


1  Rumsfeld,  D.  2002.  US  Department  of  Defense  News  Briefing. 
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develop  and  present  a  full  range  of  flood  control,  coastal  restoration,  and  hurricane  protection 
measures.”  The  Louisiana  Coastal  Protection  and  Restoration  (LACPR)  project  was  created  to 
develop  coastwide,  comprehensive  plans  to  conduct  these  activities.  The  following  case  study 
highlights  the  explicit  consideration  of  two  alternative  scenarios  and  parametric  uncertainty  in 
the  coastal  restoration  components  of  this  project.  Though  this  paper  highlights  compelling 
issues  associated  with  restoration  components  of  LACPR,  it  should  not  be  considered  a  summary 
of  techniques  or  results. 

Coastal  restoration  alternatives  were  developed  at  a  coastwide  scale  and  analyzed  by  an 
interagency  Habitat  Evaluation  Team  (HET)  consisting  of  federal,  state,  and  local  partners.  The 
HET  not  only  collaboratively  developed  alternatives,  but  also  worked  side-by-side  in  analyzing 
these  alternatives.  Freshwater  flow  diversion  to  coastal  marshes  has  been  suggested  by  many 
scientists  to  be  the  most  promising  method  to  create  new  marsh  communities  and  sustain  existing 
communities  by  offsetting  losses  due  to  sea  level  rise,  subsidence,  and  coastal  erosion.  Benefits  of 
flow  diversions  are  derived  through  two  major  mechanisms:  1)  addition  of  mineral  sediments  from 
the  diversion  water  source  and  2)  addition  of  nutrients,  which  can  stimulate  marsh  vegetative 
growth  and  increase  organic  accretion. 

The  HET  required  a  model  to  assess  flow  diversion  benefits  that  would  account  for  the  quantity 
and  quality  of  wetlands  throughout  the  study  area  over  a  50-year  planning  horizon,  but  time 
constraints  prevented  the  development  and  application  of  a  completely  mechanistic,  spatially 
explicit  tool.  A  spreadsheet  model  accounting  for  both  organic  and  inorganic  diversion  benefits  on 
an  annual  time  scale  had  been  previously  applied  under  the  Coastal  Wetland  Planning,  Protection, 
and  Restoration  Act  (CWPPRA;  Boustany  2010).  However,  the  limited  temporal  resolution 
(annual  averages)  of  the  CWPPRA  model  was  of  concern  due  to  variability  in  inputs  and 
operational  criteria  on  finer  time  scales.  Several  other  model  limitations  were  identified,  so  the 
CWPPRA  model  was  adapted  to  the  study  needs  by  improving  several  model  features,  including 
sediment  retention  and  consolidation  calculations  and  adding  intra-annual  temporal  variability  in 
hydrologic  and  sediment  inputs.  The  revised  model  was  named  the  Diversion  Benefits  Assessment 
Tool  (DBAT)1 2.  Model  alterations  were  critical  to  calculating  flow  diversion  benefits  with  respect 
to  diversion  location,  magnitude,  structure  type,  and  operation  (USACE  2009b). 

Sensitivity  analysis  to  sea  level  rise  scenarios.  Scenario  analysis  was  applied  to  quantita¬ 
tively  assess  the  sensitivity  of  decisions  to  plausible  alternative  future  conditions.  Because  Coastal 
Louisiana  is  virtually  without  vertical  relief,  small  changes  in  sea  level  exert  a  disproportionately 
large  influence  on  ecosystem  processes.  Thus,  two  sea  level  rise  scenarios  were  incorporated  into 
the  project  planning:  1)  a  low  sea  level  rise  projection  made  by  the  International  Panel  on  Climate 
Change  based  on  historic  rates  and  2)  a  high  projection  made  by  the  National  Research  Council  . 
Figure  1  presents  an  example  of  multiple  outcomes  from  restoration  alternatives  under  two  sea 
level  rise  scenarios.  By  conducting  scenario  analysis,  decision  makers  were  able  to  examine 
potential  outcomes  of  restoration  based  on  two  alternative  futures.  If  the  same  alternatives  are 


1  Subsequent  to  the  case  study  application  described  in  this  technical  note,  the  DBAT  was  further  revised  and 
renamed  the  Sediment  and  Nutrient  Diversion  (SAND)  model.  At  publication,  the  SAND  model  was  undergoing 
final  testing  prior  to  submittal  for  certification  as  a  USACE  Planning  Model. 

2  Note  that  current  policy  for  addressing  uncertainties  in  SLR  should  be  used  and  may  differ  from  the  approach  used 
in  this  case  study.  At  publication,  current  policy  is  addressed  in  Engineer  Circular  (EC)  1165-2-212  (USACE  2011). 
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identified  regardless  of  scenario,  then  decision  makers  can  have  greater  confidence  in  the  decision. 
Figure  1  shows  that  five  alternatives  are  consistently  ranked  for  both  sea  level  rise  scenarios. 


Figure  1 .  Sample  plan  comparison  for  two  sea  level  rise  scenarios  (SLR  indicates  higher  sea  level  rise). 


Parametric  uncertainty  analysis.  The  model  utilized  numerous  physical  and  ecological 
variables  (Table  1).  Existing  data  and  literature  sources  were  used  to  parameterize  the  model, 
requiring  professional  judgment  in  cases  with  conflicting  or  limited  data.  Given  the  inherent 
uncertainty  in  many  of  the  model  variables,  a  parametric  uncertainty  analysis  was  conducted 
using  parameters  in  Table  1. 

Uncertainty  about  the  true  value  of  model  inputs  was  described  using  probability  distributions 
for  each  parameter  developed  from  a  combination  of  literature  review  and  HET  professional 
judgment.  Each  parameter  was  assigned  a  range  of  values  that  were  then  randomly  sampled, 
combined,  and  analyzed  using  the  model  and  the  probability  distribution  function  (PDF)  for  each 
variable.  A  Monte  Carlo  analysis  was  performed  in  Microsoft  Excel  using  ten  thousand 
combinations  of  input  values  for  model  variables  based  upon  a  random  sampling  of  the  PDFs. 
Input  sets  were  computed  for  a  select  study  site  (Figure  2).  Understanding  parametric  uncertainty 
is  crucial  to  the  development  of  any  ecological  model  (Schmolke  et  al.  2010),  and  this  analysis 
informed  the  HET  of  the  potential  impacts  of  this  form  of  uncertainty  on  model  results. 
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Table  1.  Parameters  for  the  adapted  freshwater  diversion  model. 

Parameter  (units) 

Source 

Initial  land  and  project  area  (ac) 

Available  data 

Average  water  depth  and  width  (ft) 

Data  when  available  and  professional  judgment 

Maximum  tidal  velocity  (ft/s) 

Data  when  available  and  professional  judgment 

Roughness  height  (ft) 

Data  when  available  and  professional  judgment 

Land  loss  rate  (%/year) 

Available  data  (USGS1 978-1 990) 

Bulk  density  (g/cmJ) 

Data  coupled  with  professional  judgment 

Sediment  rating  -  intercept  and  exponent 

Snedden  et  al.  (2007) 

Size  fraction  -  sand,  silt,  clay,  flocculants  (%) 

Snedden  et  al.  (2007) 

Fall  velocity  -  sand,  silt,  clay,  flocculants 
(ft/s) 

Calculated  based  on  Soulsby  (1997) 

Plant  productivity  rate  (g/m2*yr) 

Professional  judgment  informed  by  Gosselink  (1984), 

Nyman  et  al.  (1995),  Visser  et  al.  (2004),  and  other  sources. 

Percent  of  plant  biomass  that  is  N  and  P  (%) 

Foote  and  Reynolds  (1997) 

Background  concentration  of  N  and  P  (mg/L) 

Hyfield  (2004)  i 

Sourcewater  concentration  of  N  and  P 
(mg/L) 

Available  data 

Nutrient  retention  (%) 

Professional  judgment 

Figure  2.  Example  of  the  parametric  uncertainty  analysis  results  showing  the  predicted  marsh  acreage 
for  the  Future  Without  Project  condition  bounded  by  confidence  bands. 
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DISCUSSION:  Historically,  quantitative  risk  assessment  methods  (such  as  sensitivity  and 
uncertainty  analysis)  have  created  a  challenging  computational  hurdle  for  practical 
implementation,  but  with  increased  computer  power  and  readily  available  software,  the  obstacle 
is  fading  away  (Ascough  et  al.  2008).  The  example  presented  in  this  technical  note  describes  an 
interagency  effort  to  assess  the  implications  of  uncertainty  on  environmental  benefits  for  a 
coastal  restoration  project  in  Louisiana.  All  of  the  analyses  were  conducted  using  Microsoft 
Excel,  and  required  just  a  few  days  of  effort.  Discussions  among  the  HET  regarding  the  source 
and  magnitude  of  uncertainties  and  associated  risks  helped  inform  later  decisions  regarding 
alternative  formulation,  alternative  comparison,  and  project  benefit  analyses. 

Quantitative  risk  assessment  can  include  a  variety  of  methods  not  employed  for  this  case  study, 
and  a  more  detailed  discussion  of  the  subject  is  provided  by  Suedel  et  al.  (2012).  The  selection  of 
an  appropriate  strategy  for  addressing  uncertainty  is  case-specific.  However,  nearly  every  study 
can  benefit  from  efforts  to  identify,  characterize,  and  quantify  the  primary  sources  of  model 
uncertainty.  As  demonstrated  by  this  case  study,  such  efforts  are  easily  implemented  and  the 
ability  of  these  analyses  to  inform  decision  making  justifies  their  use. 

ADDITIONAL  INFORMATION:  Research  presented  in  this  technical  note  was  developed  under 
the  Enviromnental  Benefits  Analysis  (EBA)  Research  Program.  The  US  ACE  proponent  for  the 
EBA  Program  is  Rennie  Sherman  and  the  Technical  Director  is  Dr.  Al  Cofrancesco.  Technical 
reviews  were  provided  by  Dr.  Burton  Suedel,  Dr.  Kelly  Burks-Copes  (ERDC  Environmental 
Laboratory),  Dr.  Chuck  Theiling  (USACE  Rock  Island),  and  Sandra  Stiles  (USACE  New  Orleans 
District)  are  gratefully  acknowledged. 

For  additional  information,  contact  the  authors,  S.  Kyle  McKay  (601-415-7160,  Kyle.McKay(d), 
usace.army.mil )  or  Dr.  Craig  Fischenich  (601-634-3449,  Craig.  J.Fischenich(d),usace. army. mil),  or 
the  manager  of  the  Ecosystem  Management  and  Restoration  Research  Program,  Glenn  Rhett  (60 1  - 
634-3717,  Glenn .  G. Rhett/wyisace. army. m U ) .  This  technical  note  should  be  cited  as  follows: 

McKay,  S.K.  and  J.C.  Fischenich.  2014.  Uncertainty  and  sensitivity  of  ecosystem 
restoration  decisions:  A  case  study  from  Coastal  Louisiana.  EMRRP  Technical 
Notes  Collection.  ERDC  TN-EMRRP-EBA-21.  Vicksburg,  MS:  US  Army 
Engineer  Research  and  Development  Center.  http://cw-environment.usace. 
army.mil/eba/ 
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